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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timery. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)13 Responsive to communication(s) filed on 28 December 2001 . 
2a)D This action is FINAL. 2b)[X] This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) E3 Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-20 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) [3 The drawing(s) filed on 28 December 2001 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 

1 . Claims 1 -20 are pending in this office action. 

Information Disclosure Statement 

2. The information disclosure statements (IDS) submitted on 12/28/2001 and 
05/21/2001 have been considered by the examiner. 

Claim Objections 

3. Claims 12-14 are objected to because of the following informalities: Claim 12 is 
written as being dependent on claim 12: The examiner obviously believes the intent 
was to have claim 12 be dependent on claim 1 1 . Please make the appropriate 
correction and check the remaining dependent claims for accuracy in terms of 
dependency. Appropriate correction is required. 

4. Claims 1 6-1 9 are objected to because of the following informalities: Claim 1 6 is 
written as being dependent on claim 16. The examiner obviously believes the intent 
was to have claim 16 be dependent on claim 15. Please make the appropriate 
correction and check the remaining dependent claims for accuracy in terms of 
dependency. Appropriate correction is required. 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1-6, 8, 10-13, 15-18 and 20 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over U.S. Provisional Application 60/213562 by Rudder et al. filed on June 
22, 2000 (hereinafter Rudder), in view of U.S. Patent 5,479,476 by Finke-Anlauff 
(hereinafter Finke). 

7. Note: US Patent Application Publication 2002/0059425 claims priority to 
Provisional Application 60/213562. 

8. With respect to Claim 1 , Rudder teaches a multimodal communication method 
comprising; accessing a multimodal profile that contains at least multimodal preference 
information associated with at least one input modality and at least one output modality 
and multimodal preference information (Page 34, 2 nd and 3 rd paragraphs); and 
configuring at least one multimodal communication apparatus for a multimodal 
communication session based on the accessed multimodal preference information 
(Page 33, last paragraph; Page 34, starting at the 2 nd paragraph to Page 35). Rudder 
does not explicitly disclose the used of an identifier associated with the multimodal 
preference information. Finke teaches a similar method of using a profile to configure a 
communication apparatus (Col. 1 line 61 - Col. 2 line 12). The teachings of Finke 
include the use of an identifier that is associated with both input and output preference 
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information (Col. 3 line 57 - Col. 4 line 12 and Col. 5 lines 3-67, also see Fig. 3). This 
allows a user to easily select a mode of operation associated with an identifier based on 
the particular situation of the user (Col. 5 lines 63-67). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to take the method 
disclosed by Rudder and modify it as indicated by Finke such that the method further 
comprises an identifier associated with the multimodal preference information. One 
would be motivated to have this, as there is need for facilitating a users adjustment of 
operating characteristics in order to get the best performance appropriate to a current 
situation (Col. 1 lines 46-58 and Col. 5 lines 63-67 of Finke). 

9. With respect to Claim 2, Rudder in view of Finke teaches all the limitations of 
Claim 1 and further teaches wherein the step of configuring the at least one multimodal 
communication apparatus for the multimodal communication session based on the 
accessed multimodal preference information includes using the at least one identifier to 
select one of a plurality of stored multimodal preferences from a multimodal profile (Col. 
3 line 57 - Col. 4 line 12 and Col. 5 lines 3-67, also see Fig. 3 of Finke). 

10. With respect to Claim 3, Rudder in view of Finke teaches all the limitations of 
Claim 2 and further teaches configuring at least one multimodal server for a multimodal 
communication session based on the selected multimodal preference information Page 
34, starting at the 2 nd paragraph to Page 35 of Rudder). 

1 1 . With respect to Claim 4, Rudder in view of Finke teaches all the limitations of 
Claim 1 and further teaches storing a plurality of multimodal preferences for a plurality 
of different modalities to create a multimodal profile (Page 33, last paragraph; Page 34, 
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starting at the 2 nd paragraph to Page 35 of Rudder; and Col. 3 line 57 - Col. 4 line 12 
and Col. 5 lines 3-67, also see Fig. 3 of Finke). 

12. With respect to Claim 5, Rudder in view of Finke teaches all the limitations of 
Claim 1 and further teaches creating at least one multimodal profile by: presenting a 
user interface that receives input and output modality preference data to define differing 
multimodal preference information for a plurality of multimodal communication scenarios 
(Page 33, last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; 
Page 37, last paragraph to Page 38 2 nd paragraph) associated with a plurality of 
identifiers (Col. 5 lines 3-67 of Finke). 

13. With respect to Claim 6, Rudder in view of Finke teaches all the limitations of 
Claim 1 and further teaches the identifier associated with the multimodal preference 
information represents an environmental situation associated with a multimodal 
communication (Col. 3 line 57 - Col. 4 line 12 and Col. 5 lines 3-67, also see Fig. 3 of 
Finke). 

14. With respect to Claim 8, Rudder in view of Finke teaches all the limitations of 
Claim 5 and further teaches wherein the multimodal preference information includes at 
least one of; session preference information, media preference information including 
format identifiers to facilitate control of a format of information sent or received during 
the multimodal communication, and input output modality preference data (Page 33, last 
paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, last 
paragraph to Page 38 2 nd paragraph of Rudder). 
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15. With respect to Claim 10, Rudder in view of Finke teaches all the limitations of 
Claim 1 and further teaches wherein available multimodal preference information is 
based on operational capabilities of at least one of: multimodal communication 
apparatus capabilities, communication network capabilities, ambient conditions, a 
server that is accessed by the multimodal communication apparatus and a service 
accessed by the multimodal communication apparatus (Page 33, last paragraph; Page 
34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, last paragraph to Page 
38 2 nd paragraph). 

16. With respect to Claim 1 1 , Rudder teaches a multimodal profile generator 
operative to access a multimodal profile that contains at least one of multimodal 
preference information associated with at least one input modality and at least one 
output modality and multimodal preference information (Page 34, 2 nd and 3 rd 
paragraphs); and a multimodal communication apparatus configuration controller, 
operatively responsive to accessed multimodal preference information, to configure a 
multimodal communication apparatus for a multimodal communication session based 
on the accessed multimodal preference information (Page 33, last paragraph; Page 34, 
starting at the 2 nd paragraph to Page 35). Rudder does not explicitly disclose the used 
of an identifier associated with the multimodal preference information. Finke teaches a 
similar method of using a profile to configure a communication apparatus (Col. 1 line 61 
- Col. 2 line 12). The teachings of Finke include the use of an identifier that is 
associated with both input and output preference information (Col. 3 line 57 - Col. 4 line 
12 and Col. 5 lines 3-67, also see Fig. 3). This allows a user to easily select a mode of 
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operation associated with an identifier based on the particular situation of the user (Col. 
5 lines 63-67). It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to take the apparatus disclosed by Rudder and modify it as 
indicated by Finke such that the apparatus further comprises an identifier associated 
with the multimodal preference information. One would be motivated to have this, as 
there is need for facilitating a users adjustment of operating characteristics in order to 
get the best performance appropriate to a current situation (Col. 1 lines 46-58 and Col. 
5 lines 63-67 of Finke). 

1 7. With respect to Claim 1 2, Rudder teaches all the limitations of Claim 1 1 and 
further teaches a user interface, operatively coupled to the multimodal profile generator, 
that receives input and output modality preference data to define differing multimodal 
preference information for a plurality of multimodal communication scenarios (Page 33, 
last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, 
last paragraph to Page 38 2 nd paragraph) associated with a plurality of identifiers (Col. 5 
lines 3-67 of Finke); and memory, operatively coupled to the multimodal profile 
generator, that stores received input and output modality preference data and an 
associated identifier that has been associated with at least a pair of received input and 
output modality preference data, as part of the multimodal profile (Page 33, last 
paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, last 
paragraph to Page 38 2 nd paragraph) and (Col. 5 lines 3-67 of Finke). 

18. With respect to Claim 13, Rudder teaches all the limitations of Claim 12 and 
further teaches wherein the multimodal communication apparatus configuration 
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controller configures the multimodal communication apparatus for the multimodal 
communication session based on the accessed multimodal preference information 
using the at least one identifier to select one of a plurality of stored multimodal 
preferences from a multimodal profile (Col. 3 line 57 - Col. 4 line 12 and Col. 5 lines 3- 
67, also see Fig. 3 of Finke). 

1 9. With respect to Claim 1 5, Rudder teaches a multimodal communication system 
comprising: (a) a multimodal communication apparatus having: multimodal profile 
generator operative to access a multimodal profile that contains at least one of 
multimodal preference information associated with at least one input modality and at 
least one output modality and multimodal preference information (Page 34, 2 nd and 3 rd 
paragraphs); and a multimodal communication apparatus configuration controller, 
operatively responsive to accessed multimodal preference information, to configure a 
multimodal communication apparatus for a multimodal communication session based 
on the accessed multimodal preference information (Page 33, last paragraph; Page 34, 
starting at the 2 nd paragraph to Page 35); and (b) a multimodal network element, 
operatively coupled to the multimodal communication apparatus, to provide information 
during a session with the multimodal communication apparatus (Page 34, starting at the 
2 nd paragraph to Page 35). Rudder does not explicitly disclose the used of an identifier 
associated with the multimodal preference information. Finke teaches a similar method 
of using a profile to configure a communication apparatus (Col. 1 line 61 - Col. 2 line 
12). The teachings of Finke include the use of an identifier that is associated with both 
input and output preference information (Col. 3 line 57 - Col. 4 line 12 and Col. 5 lines 3- 
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67, also see Fig. 3). This allows a user to easily select a mode of operation associated 
with an identifier based on the particular situation of the user (Col. 5 lines 63-67). It 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to take the system disclosed by Rudder and modify it as indicated by Finke such 
that the system further comprises an identifier associated with the multimodal 
preference information. One would be motivated to have this, as there is need for 
facilitating a users adjustment of operating characteristics in order to get the best 
performance appropriate to a current situation (Col. 1 lines 46-58 and Col. 5 lines 63-67 
of Finke). 

20. With respect to Claim 16, Rudder teaches all the limitations of Claim 15 and 
further teaches including memory, operatively coupled to the multimodal communication 
apparatus, the memory containing received input and output modality preference data 
and an associated identifier that has been associated with at least a pair of received 
input and output modality preference data, as part of the multimodal profile (Page 33, 
last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, 
last paragraph to Page 38 2 nd paragraph) and (Col. 5 lines 3-67 of Finke). 

21. With respect to Claim 17, Rudder teaches all the limitations of Claim 16 and 
further teaches including memory, operatively coupled to the multimodal network 
element, the memory containing received input and output modality preference data and 
an associated identifier that has been associated with at least a pair of received input 
and output modality preference data, as part of the multimodal profile (Page 33, last 
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paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, last 
paragraph to Page 38 2 nd paragraph) and (Col. 5 lines 3-67 of Finke). 

22. With respect to Claim 18, Rudder teaches all the limitations of Claim 16 and 
further teaches a user interface, operatively coupled to the multimodal profile generator, 
that receives input and output modality preference data to define differing multimodal 
preference information for a plurality of multimodal communication scenarios (Page 33, 
last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; Page 37, 
last paragraph to Page 38 2 nd paragraph) associated with a plurality of identifiers (Col. 5 
lines 3-67 of Finke). 

23. With respect to. Claim 20, Rudder teaches all the limitations of Claim 15 and 
further teaches wherein the multimodal profile is transferred between the multimodal 
communication apparatus and the multimodal network element (Page 34, starting at the 
2 nd paragraph to Page 35 of Rudder; Page 37, last paragraph to Page 38 2 nd 
paragraph). 

24. Claims 7, 9, 14 and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rudder in view of Finke as applied to claim 1 above, and further in 
view of U.S. Patent 5,983,186 by Miyazawa et al. (Miyazawa). 

25. With respect to Claim 7, Rudder in view of Finke teaches all the limitations of 
Claim 1 and further teaches multimodal preference information may be associated with 
the user's state and context as well as conditions of the apparatus being used (Page 33, 
last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; and Page 
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38 2 nd paragraph; of Rudder). Rudder in view of Finke does not explicitly disclose the 
multimodal preference information includes ambient condition threshold data associated 
with at least one identifier. Miyazawa teaches the use of ambient condition threshold 
data associated with operational settings (Col. 5 lines 8-49). Ambient threshold data is 
compared with a detected ambient condition level (Col. 12 line 62 - Col. 13 line 23). 
Selection of settings for the apparatus is based on the comparison (Col. 12 line 62 - Col. 
13 line 23). This allows for improved interaction by, for example, making it easier to 
hear even when ambient noise is present (Col. 13 lines 1 8-22). It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to take the 
method disclosed by Rudder in view of Finke and modify it as indicated by Miyazawa 
such that the method further comprises wherein the multimodal preference information 
includes ambient condition threshold data associated with at least one identifier. One 
would be motivated to have this, as there is need for systems and methods that 
facilitate greater and more consistent user interaction (Page 5 of Rudder). 

26. With respect to Claim 9, Rudder in view of Finke and in further view of Miyazawa 
teaches all the limitations of Claim 7 and further teaches detecting an ambient condition 
level associated with the multimodal communication apparatus, comparing the ambient 
condition level to the ambient condition threshold data; and selecting a multimodal input 
and output setting for the multimodal communication apparatus based on the 
comparison (Col. 5 lines 8-49 and Col. 12 line 62- Col. 13 line 23 of Miyazawa). 

27. With respect to Claim 14, Rudder in view of Finke teaches all the limitations of 
Claim 12 and further teaches multimodal input and output settings may be selected 
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based on the user's state and context as well as conditions of the apparatus being used 
(Page 33, last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 of Rudder; 
and Page 38 2 nd paragraph; of Rudder). Rudder in view of Finke does not explicitly 
disclose the multimodal communication apparatus configuration controller detects an 
ambient condition level associated with the multimodal communication apparatus, 
compares the ambient condition level to the ambient condition threshold data; and 
makes the selection based on this setting. Miyazawa teaches the use of ambient 
condition threshold data associated with operational settings (Col. 5 lines 8-49). 
Ambient threshold data is compared with a detected ambient condition level associated 
with the apparatus (Col. 12 line 62 - Col. 13 line 23). Selection of settings for the 
apparatus is based on the comparison (Col. 12 line 62 - Col. 13 line 23). This allows for 
improved interaction by, for example, making it easier to hear even when ambient noise 
is present (Col. 13 lines 18-22). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to take the apparatus disclosed by Rudder in 
view of Finke and modify it as indicated by Miyazawa such that the apparatus further 
comprises wherein the multimodal communication apparatus configuration controller 
detects an ambient condition level associated with the multimodal communication 
apparatus, compares the ambient condition level to the ambient condition threshold 
data; and selects a multimodal input and output setting for the multimodal 
communication apparatus based on the comparison. One would be motivated to have 
this, as there is need for systems and methods that facilitate greater and more 
consistent user interaction (Page 5 of Rudder). 
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28. With respect to Claim 19, Rudder in view of Finke teaches all the limitations of 
Claim 16 and further teaches further teaches multimodal input and output settings may 
be selected based on the user's state and context as well as conditions of the apparatus 
being used (Page 33, last paragraph; Page 34, starting at the 2 nd paragraph to Page 35 
of Rudder; and Page 38 2 nd paragraph; of Rudder). Rudder in view of Finke does not 
explicitly disclose the multimodal communication apparatus configuration controller 
detects an ambient condition level associated with the multimodal communication 
apparatus, compares the ambient condition level to the ambient condition threshold 
data; and selects a multimodal input and output setting for the multimodal 
communication apparatus based on the comparison. Miyazawa teaches the use of 
ambient condition threshold data associated with operational settings (Col. 5 lines 8-49). 
Ambient threshold data is compared with a detected ambient condition level associated 
with the apparatus (Col. 12 line 62 - Col. 13 line 23). Selection of settings for the 
apparatus is based on the comparison (Col. 12 line 62 - Col. 13 line 23). This allows for 
improved interaction by, for example, making it easier to hear even when ambient noise 
is present (Col. 13 lines 18-22). It would have been obvious to one of ordinary skill in 
the art at the time the system was made to take the apparatus disclosed by Rudder in 
view of Finke and modify it as indicated by Miyazawa such that the system further 
comprises wherein the multimodal communication apparatus configuration controller: 
detects an ambient condition level associated with the multimodal communication 
apparatus, compares the ambient condition level to the ambient condition threshold 
data; and selects a multimodal input and output setting for the multimodal 
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communication apparatus based on the comparison. One would be motivated to have 
this, as there is need for systems and methods that facilitate greater and more 
consistent user interaction (Page 5 of Rudder). 



Conclusion 

29. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

30. U.S. Patent 5,918,222 by Fukui et al. "Information disclosing apparatus and 
multi-modal information input/output system" June 29, 1999. Discloses a mechanism 
for responding to a demand for information in the most appropriate manner based in 
part on a response rule set that takes into account personal relationship information 
(See Fig. 11). 

31 . U.S. Patent 6,285,891 by Hoshino et al. "Radio communication apparatus having 
a plurality of communication functions" September 4, 2001. Discloses selectable 
operation modes suitable for various environments. The operation modes determine 
how the various functions of the apparatus will behave (See Fig. 2). 

32. U.S. Patent 6,377,91 3 by Coffman et al. "Method and System for Multi-Client 
access to Dialog System" April 23, 2002. Discloses multi-modal system that uses a 
multi-modal history to determine the intent of a input. Also makes use of profile for 
determining output device preference. 

33. U.S. Patent 6,829,603 by Chai et al. "System, method and program product for 
interactive natural dialog" December 7, 2004. In general, discloses a system with one 
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ore more modalities of interaction, which may be chosen based on user specified 
preferences. Does not disclose the details of such a mechanism. 

34. U.S. Patent Application Publication 2001/0047263 by Smith et al. "Multimodal 
user interface" November 29, 2001. Suggests the user may choose the mode of 
entering commands based on environment, for example, using speech while in the car. 
Does not disclose a profile mechanism as claimed. 

35. U.S. Patent Application Publication 2002/0059425 by Belifore et al. "Distributed 
Computing Services Platform" May 16, 2002. Claims priority to the relied upon 
provisional application 60/21 3,562. 

36. U.S. Patent Application Publication 2002/0152255 by Smith, Jr. et al. 
"Accessibility on Demand" October 17, 2002. Discloses accessibility profiles created for 
different environmental conditions that determine preferred settings for input and output 
modalities. 

37. W3C, Hickey, Marianne (editor) "Multimodal Requirements for Voice Markup 

. Languages" July 10, 2000, W3C Working Draft available www.w3.org/TR/multimodal- 
reqs. Discloses the suggestion for determining available modalities and further make 
use of modalities based upon environmental factors (like noise) and user controlled 
factors. (See section 4.4 particularly). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David Lazaro whose telephone number is 571-272- 
3986. The examiner can normally be reached on 8:30-5:00 M-F. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Saleh Najjar can be reached on 571-272-4006. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



David Lazaro 
June 24, 2005 




